Combining Multi-Scale Character Recognition and Linguistic Knowledge for Natural Scene Text OCR

International audienceUnderstanding text captured in real-world scenes is a challenging problem in the field of visual pattern recognition and continues to generate a significant interest in the OCR (Optical Character Recognition) community. This paper proposes a novel method to recognize scene texts avoiding the conventional character segmentation step. The idea is to scan the text image with multi-scale windows and apply a robust recognition model, relying on a neural classification approach, to every window in order to recognize valid characters and identify non valid ones. Recognition results are represented as a graph model in order to determine the best sequence of characters. Some linguistic knowledge is also incorporated to remove errors due to recognition confusions. The designed method is evaluated on the ICDAR 2003 database of scene text images and outperforms state-of-the-art approaches

Text Recognition in Multimedia Documents: A Study of two Neural-based OCRs Using and Avoiding Character Segmentation

International audienceText embedded in multimedia documents represents an important semantic information that helps to automatically access the content. This paper proposes two neural-based OCRs that handle the text recognition problem in different ways. The first approach segments a text image into individual characters before recognizing them, while the second one avoids the segmentation step by integrating a multi-scale scanning scheme that allows to jointly localize and recognize characters at each position and scale. Some linguistic knowledge is also incorporated into the proposed schemes to remove errors due to recognition confusions. Both OCR systems are applied to caption texts embedded in videos and in natural scene images and provide outstanding results showing that the proposed approaches outperform the state-of-the-art methods

Combinaison d'approches neuronales et de connaissances linguistiques pour la reconnaissance de texte dans les documents multimédias

This thesis focuses on the recognition of textual clues in images and videos. In this context, OCR (optical character recognition) systems, able to recognize caption texts as well as natural scene texts captured anywhere in the environment have been designed. Novel approaches, robust to text variability (differentfonts, colors, sizes, etc.) and acquisition conditions (complex background, non uniform lighting, low resolution, etc.) have been proposed. In particular, two kinds of methods dedicated to text recognition are provided:- A segmentation-based approach that computes nonlinear separations between characters well adapted to the localmorphology of images;- Two segmentation-free approaches that integrate a multi-scale scanning scheme. The first one relies on a graph model, while the second one uses a particular connectionist recurrent model able to handle spatial constraints between characters.In addition to the originalities of each approach, two extra contributions of this work lie in the design of a character recognition method based on a neural classification model and the incorporation of some linguistic knowledge that enables to take into account the lexical context.The proposed OCR systems were tested and evaluated on two datasets: a caption texts video dataset and a natural scene texts dataset (namely the public database ICDAR 2003). Experiments have demonstrated the efficiency of our approaches and have permitted to compare their performances to those of state-of-the-art methods, highlighting their advantages and limits.Les travaux de cette thèse portent sur la reconnaissance des indices textuels dans les images et les vidéos. Dans ce cadre, nous avons conçu des prototypes d'OCR (optical character recognition) capables de reconnaître tant des textes incrustés que des textes de scène acquis n'importe où au sein d'images ou de vidéos. Nous nous sommes intéressée à la définition d'approches robustes à la variabilité des textes et aux conditions d'acquisition. Plus précisément, nous avons proposé deux types de méthodes dédiées à la reconnaissance de texte : - une approche fondée sur une segmentation en caractères qui recherche des séparations non linéaires entre les caractères adaptées à la morphologie de ces derniers ; - deux approches se passant de la segmentation en intégrant un processus de scanning multi-échelles ; la première utilise un modèle de graphe pour reconnaître les textes tandis que la seconde intègre un modèle connexionniste récurrent spécifiquement développé pour gérer les contraintes spatiales entre les caractères.Outre les originalités de chacune des approches, deux contributions supplémentaires de ce travail résident dans la définition d'une reconnaissance de caractères fondée sur un modèle de classification neuronale et l'intégration de certaines connaissances linguistiques permettant de tirer profit du contexte lexical. Les différentes méthodes conçues ont été évaluées sur deux bases de documents : une base de textes incrustés dans des vidéos et une base publique de textes de scène. Les expérimentations ont permis de montrer la robustesse des approches et de comparer leurs performances à celles de l'état de l'art, mettant en évidence leurs avantages et leurs limites

A Comprehensive Neural-Based Approach for Text Recognition in Videos using Natural Language Processing

International audienceThis work aims at helping multimedia content understanding by deriving bene t from textual clues embedded in digital videos. For this, we developed a complete video Optical Character Recognition system (OCR), speci cally adapted to detect and recognize embedded texts in videos. Based on a neural approach, this new method outperforms related work, especially in terms of robustness to style and size variabilities, to background complexity and to low resolution of the image. A language model that drives several steps of the video OCR is also introduced in order to remove ambiguities due to a local letter by letter recognition and to reduce segmentation errors. This approach has been evaluated on a database of French TV news videos and achieves an outstanding character recognition rate of 95%, corresponding to 78% of words correctly recognized, which enables its incorporation into an automatic video indexing and retrieval system

Reconnaissance automatique de textes dans les vidéos à l’aide d’un OCR et de connaissances linguistiques

National audienceOur work aims at helping multimedia content understanding by extracting textual clues embedded in digital video data. For this, wedeveloped a video Optical Character Recognition (OCR) system, specifically adapted to detect and recognize embedded texts. Based on a neural approach, our method outperforms related work especially in terms of robustness to style and size variability, to background complexity and to low resolution of the image. We also introduced a language model that drives several steps of the video OCR in order to remove ambiguities related to recognition and reduce segmentation errors. This approach has been evaluated on a database of French TV news videos and achieves a character recognition rate of 95%, which enables its incorporation in a video indexing system

Text Recognition in Videos using a Recurrent Connectionist Approach

International audienceMost OCR (Optical Character Recognition) systems developed to recognize texts embedded in multimedia documents segment the text into characters before recognizing them. In this paper, we propose a novel approach able to avoid any explicit character segmentation. Using a multi-scale scanning scheme, texts extracted from videos are first represented by sequences of learnt features. Obtained representations are then used to feed a connectionist recurrent model specifically designed to take into account dependencies between successive learnt features and to recognize texts. The proposed video OCR evaluated on a database of TV news videos achieves very high recognition rates. Experiments also demonstrate that, for our recognition task, learnt feature representations perform better than hand-crafted features

Journal of Cardiovascular Magnetic Resonance BioMed Central Poster presentation

Detection of pathological areas and estimation of viability parameters in late-enhancement cardiac MR